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Background / Context: 

The early identification of students at-risk for future reading difficulty has become a focal point 
for K-12 stakeholders seeking to actively prevent the emergence of student reading deficits. 

Early and active intervention efforts for struggling readers have taken on greater urgency given 
the accountability pressures that stem from the No Child Left Behind (NCLB, 2002) federal 
legislation and in light of the observation that reading trajectories establish early and remain 
relatively stable over time (Juel, 1988; Schatschneider et al., 2004). Interest in preventing initial 
reading difficulties from snowballing into long-term reading failure (Chard et al., 2008; Coyne & 
Ham, 2006; Torgesen, 2002) has promoted the development and implementation of the Response 
to Intervention (Rtl) instructional model. The Rtl model is an integrated framework designed to 
provide early screening of student reading performance, enable continuous monitoring of student 
reading progress, and allow active and focused intervention in cases where reading difficulties 
are observed (Fuchs, Fuchs, & Vaughn, 2008). Although practitioners and researchers continue 
to debate issues regarding how to identify students in need of supplemental instruction, structure 
and deliver instructional support, and determine the adequacy of response to strategic 
intervention (Gersten et al., 2008), Rtl-based treatment models have been endorsed by the federal 
government (IDEA, 2004) and are increasingly being adopted and implemented in elementary 
and secondary school settings (Spectrum, 2010). 

The widespread use of early screening and progress monitoring data to inform instructional 
decisions regarding at-risk students is generally viewed as more effective and equitable than the 
‘wait to fail’ IQ/achievement discrepancy model that has been historically utilized in scholastic 
contexts (Fuchs & Fuchs, 2008). Yet, ascertaining whether struggling learners respond to an 
instructional or behavioral intervention is a complex task for K-12 stakeholders. When Rtl 
models are employed, the challenge stems in part from the non-random application of 
intervention regimes that are designed to vary in response to recipient outcomes. In the Rtl 
framework, students can be entered and exited from a supplemental instructional program at 
different points in time and may also be moved to one or more alternative forms of treatment 
over the course an academic year. Individual students often have distinct assessment schedules as 
well. School-based Rtl treatment regimes are thus not unlike the individualized treatment 
approaches that are common to the health and social service fields. For example, when different 
drugs or dosages are prescribed, physical and mental health therapies are changed, and clinic 
reward structures are modified on an individualized basis to optimally align program services 
with individual needs, the timing and sequence of one or more modifications to treatment adds 
an additional complication for stakeholders charged with tracking the responsiveness of 
participants to intervention. However, a unique opportunity arises when key outcomes are 
repeatedly measured and the type and timing of programmatic changes are recorded. Under these 
conditions, researchers and practitioners can draw on the power of the associated interrupted 
time series (ITS) to increase the validity of inferences regarding the average treatment effect of 
an intervention as well as ascertain the responsiveness of individual students to intervention. 

ITS designs are often recommended as an alternative method for providing control over threats 
to internal validity (Bloom, 2003; Shadish, Cook, & Campbell, 2002). ITS designs tend to be 
minimally intrusive while becoming increasingly powerful as the number of pre and post- 
treatment observations increase, as comparison or control groups are added to the design, and as 
an abrupt change in the outcome can be demonstrated at the point at which the intervention has 
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occurred (Shadish et al., 2002). Yet, the modeling of ITS -related data can present a challenge, 
particularly when intervention schedules vary by recipient. A potential solution to capturing the 
richness and complexity of Rtl-based designs is offered by the procedural and analytic flexibility 
of multilevel models. Use of multilevel growth modeling techniques allows estimation of 
individual changes in performance level and growth coordinated with the exact time at which 
intervention is delivered. Moreover, variability in the short and longer term responses of 
recipients offers the opportunity to investigate whether variation in the type, timing, or delivery 
of one or more interventions is associated with outcome level or growth. 

Purpose / Objective / Research Question / Focus of Study: 

With respect to the challenges and opportunities associated with Rtl-based designs, the purpose 
of this paper is to present a multilevel piecewise model that estimates a staggered interrupted 
time series individualized for each participant. The demonstration draws on a unique data set that 
contains a time series of upward of 16 academic year test scores, student background 
characteristics, and the date at which a supplemental literacy intervention was implemented for 
each program participant. Two-level unconditional and conditional piecewise growth models 
were applied to the data obtained over the course of the study as a means for estimating student 
outcomes. The following research questions were investigated, 1) What was the shape of the pre 
and post intervention growth functions? 2) On average, did students’ level of literacy 
performance and rate of literacy growth increase after supplemental reading support was 
initiated? 3) Was there statistical variation in students’ response to intervention initially and over 
time? and, 4) Were one or more student and instructional characteristics predictive of the 
variation in intervention responses? 

Setting: 

The research was conducted on data obtained from a moderately- sized school district in the 
Pacific Northwest. The district serves close to 6,000 students each year. The student body is 
approximately 75% White, 14% Latino, 3% African American, 3% Asian American, 3% Native 
American, and 2% Other. In recent years, approximately 44% of district students have received a 
free or reduced-price lunch and 3% of district students have been identified as English language 
learners. 

Population / Participants / Subjects: 

The analytic sample was comprised of 155 struggling readers who began kindergarten either in 
the 2008-09 or 2009-10 school year. The overall sample was identified as 49% (N = 74) female, 
74% (N = 111) White, 19% ( N = 29) Latino, 3% ( N = 5) African American, 3% ( N = 4) Asian, 
and 1% (N = 1) Native American. Fifty-eight percent of the sample received a free or reduced 
priced lunch during kindergarten while 4% were identified as English language learners. 

Intervention / Program / Practice: 

Thirty additional minutes of daily reading instruction was offered to students not meeting 
kindergarten reading benchmarks. Each struggling reader received teacher-directed instruction in 
3-5 student small group settings. The supplemental instruction was designed to promote the 
development of basic phonemic, alphabetic, and fluency skills using the fundamental “big ideas” 
and best practices that underlie and facilitate early childhood literacy development (National 
Reading Panel, 2000). 
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Significance / Novelty of study: 

The use of piecewise modeling compliments and extends the descriptive approaches currently 
used by researchers and practitioners to track student performance and determine whether 
individual students’ response to intervention is adequate for meeting benchmark goals. By 
mapping the structure of the individualized instructional regimes delivered to struggling readers, 
estimates of the pre and post intervention growth trajectories and the amount of student-to- 
student variation in response to systematic supplemental reading instruction were obtained. 
Estimation of individual piecewise growth trajectories provides several advantages for 
researchers and practitioners who study and implement school-based Rtl instruction. Of 
particular note is the opportunity to describe the Rtl-based trajectory of change for struggling 
readers and also to model associated student-to-student variability in response to intervention. 
Modeling the variability in intercept and slope using piecewise growth models potentially 
enables one to determine which features of instruction or characteristics of students are 
associated with greater or lesser short and long term responses. The specificity with which 
individualized instructional regimes can be identified and modeled is argued to maximize the use 
of Rtl data, facilitate the estimation of treatment effects, allow correct identification of patterns 
of inadequate response, and enable educational leaders to use limited resources optimally. 

Statistical, Measurement, or Econometric Model: 

Two-level unconditional and conditional piecewise growth models were applied to the data. 
Specification of the piecewise growth models presented a challenge as the timing of treatment 
and the duration of particular instructional supplements were purposely differentiated by student. 
Specifically, supplemental instruction was initiated with different students for different durations 
at different points in time during the academic year. To accommodate the structure of the 
individualized Rtl instructional model, the number of elapsed instructional weeks prior to the 
start and for the duration of the supplemental instruction for each student was calculated by 
matching intervention dates with the academic year calendar. Taking this approach, individual 
students’ differential instructional exposure (at particular assessment points) was explicitly 
represented in the model through the inclusion of parameters that captured the intervention start 
point and the duration of supplemental instruction. Equation 1 specifies the basic form of the 
unconditional piecewise growth model. In equation 1, it can be seen that student performance is 
conceived as a function of one status and four growth or change parameters. All growth model 
parameters were specified to vary randomly across individuals. 

Y,i = Koi + TCu(Pre-Slope)+ njJTx-Level )+ nv, (Post-Slope) + npfPost-Slope ) + e t i ( 1 ) 

In the piecewise model, time was represented both as a continuous variable capturing the linear 
and quadratic change in student literacy across multiple assessments during the kindergarten year 
and as a dichotomous variable (coded 0, 1) capturing the difference in literacy performance at the 
onset of supplement instruction. The coding used to represent the change in literacy across the 
different time periods defines the status parameter (nod as the expected performance of student i 
just prior to the start of the intervention. After estimating the parameters of the unconditional 
model, variation in the literacy status (nod and growth rates (tt//, tt?;, n;u, i ud of students was 
modeled conditionally as a function of student and instructional characteristics. 

Usefulness / Applicability of Method: 
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Staggered ITS models are relevant when outcomes are repeatedly measured and treatment 
regimes are individual specific. To apply, coding schemes individualized to reflect the treatment 
regime experienced by specific individuals need to be specified. The staggered ITS model is then 
applicable to any educational, social, or health context where repeated measurements are 
obtained on individuals and the timing, nature, and duration of an intervention is recorded. 

Research Design: 

The study is based on a quasi-experimental (staggered) interrupted time series design 

Data Collection and Analysis: 

Letter naming fluency scores (LNF; Good & Kaminski, 2002), student characteristics, and 
intervention data were obtained from the administrative records of a Pacific Northwest school 
district. Piecewise growth models were applied to the time series data. 

Findings / Results: 

Estimation of the unconditional piecewise model revealed that prior to the start of the 
intervention, students were gaining .8 letter names per week and were able to identify 2.2 letter 
names on average. At the onset of the intervention, LNF performance remained constant (i.e., no 
change in level) but the instantaneous rate of growth doubled relative to the pre-treatment period 
(tzm = 1.50). Treatment growth was also observed to decelerate at a rate of .02 letters per week 
over the remainder of the kindergarten year. Variance estimates indicated that there were 
statistically significant student-to-student differences in LNF status and growth prior to the start 
of the intervention as well as in all of the treatment -related growth and change parameters. To 
illustrate the variability in LNF outcomes, Figure 1 presents the growth trajectories for students 
who began to receive supplemental instruction during week 6 of the kindergarten year (insert 
Table 1 and Figure 1 here). Conditional model results indicated that LNF status at the start of the 
intervention and the timing of the start of treatment were associated with each of the treatment- 
related growth parameters. Initially higher performing students had a larger initial LNF status 
change, faster linear LNF growth, and greater LNF deceleration relative to their peers with 
initially lower levels of letter naming fluency. Students who began treatment during the second 
half of the school year experienced an initial drop in LNF at the onset of treatment, had relatively 
lower instantaneous LNF growth, but had greater LNF acceleration than their peers (see Figure 2 
for examples). Demographic characteristics were generally not associated with LNF outcomes. 

Conclusions: 

Individual response to intervention may differ initially and over time. Variability in outcomes 
may be due to characteristics of the individual and/or characteristics of the treatment context. 
Beyond descriptively tracking outcomes, understanding why different individuals respond in 
different ways should be a goal of researchers and practitioners. Application of staggered ITS 
models enables analytic flexibility to model the dynamic and fluid nature of contemporary 
instructional and clinical practice and the diversity and individualization of treatment for students 
at different points in time. Nonetheless, while the present demonstration offered a novel 
approach for modeling the wealth of data obtained in Rtl contexts, examination of additional 
data sets that include a variety of outcome measures, indicators describing the components of 
instruction and their delivery, and more diverse student samples are recommended to better 
understand the generality of these results. 
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Appendix B. Tables and Figures 

Table 1 

Unconditional and Conditional Piecewise Growth Model Results 





Pre Tx 
Slope 


Stait ofTx 
Status 


Post Tx 
Stahls A 


Post Tx 
Linear Slope A 


Post Tx 
Quadratic A 


Unconditional Model 
Intercept 


0.797 (0.097)** 


* 2.212(0.524)*** 


-1.290(0.657) 


1.501(0.152)*** 


-0.021 (0.005)*** 


Conditional Model 
Intercept 


0.743(0.103)** 


* 2.803(0.543)*** 


0.332(0.783) 


1.899(0.183)*** 


•0.032 (0.005)*** 


Cohort 
Initial Status 
Tx Start Status 
Tx Timing 


-0 005(0.007) 


-0.589(0.351) 


1.243(0.087)** 

-8.664(2.695)** 


* 0.113(0.017)*** 
-2.384(0.523)*** 


-0.003(0.001)*** 

0.064(0.029)* 



Note. A dummy code was used to identify students in the 2009-10 cohort and students who began 
treatment in the second half of the academic year. Standard errors are in the parentheses. 

*p< .05. **p< .01, ***/>< .001 
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Figure 1. Letter naming fluency as a function of the point of intervention and time 
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Figure 2. Letter naming fluency as a function of the point of intervention and time 
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